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Abstract 



Drawing primarily on the leamer-as-processor and acquisition metaphors, there is a long 
tradition of assessment and methodological practices for evaluating learning. Because the 
individual is the unit of analysis, questions are rarely asked regarding the interaction between the 
agent and environment broadly defined, leading to impoverished descriptions of the learning 
process. Further, an acquisition focus has led to an emphasis on ready-made knowledge as 
opposed to knowing-in-the-making. In contrast, in this article we adopt a situative perspective of 
what it means to know and learn, describing knowing as a continuous event distributed across 
multiple time frames and environmental particulars (e.g., textbooks, collaborating individuals, 
previous experiences, and computer representations). We advance a methodology for capturing 
cognition in situ, with the goal of tracing the historical emergence and development of practices, 
concepts, and resources, as well as the role of particular interventions in supporting this process. 
It is this latter information that is especially useful for researchers interested in carrying out 
design experiments where research findings with respect to one iteration of the course are cycled 
into the design of future course instantiations. The sociological approach of actor-network 
theory provides the structural framework within which our Constructing Networks of Activity 
(CNA) methodology is grounded. Our interest in actor-network theory is not at the sociological 
level, but we couple it with emerging theoretical perspectives of learning and cognition to create 
a new analytical tool for examining learning and the context within which it occurs. In 
advancing our CNA methodology, we adhere to a set of criteria for researchers introducing novel 
methods that was put forth by Schoenfeld (1992). In addition to setting the context and 
providing a rationale for the CNA methodology, our discussion includes an in-depth description 
of the methodology along with its application to data sets. Following these examples, a 
methodological discussion of the reliability, validity, scope of application, and limitations of the 
method is forwarded. 
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CONSTRUCTING NETWORKS OF ACTIVITY: AN IN-SITU RESEARCH 

METHODOLOGY 

INTRODUCTION 

In spite of the intuitive and theoretical appeal of situated cognition (Brown, Collins, & 
Duguid, 1989; Greeno, 1997; Lave & Wenger, 1991) and distributed cognition (Pea, 1993; 
Resnick, Levine, & Teasley, 1991; Salomon, 1993), there has been little empirical research that 
actually attempts to understand how learner understandings are constructed and are grounded in 
contextual particulars (see Barab, Hay, Barnett, & Squire, 1998; Roschelle & Clancey, 1992; 
Roth, 1996, 1998; Roth & Bowen, 1995 for some exceptions). In fact, research in general tends 
to look at the products, not the processes of learning (Wittrock & Baker, 1991 ; Young, 
Kulikowich, & Barab, 1997). Some of the difficulties with capturing the process of learning are 
only exacerbated when one adopts a situated perspective on what it means to know and learn 
(Brown, Collins, & Duguid, 1989; Lave & Wenger, 1991). This is because from this 
perspective, knowledge, more aptly phrased “knowing about” is no longer conceived of as a 
static structure residing in the individual’s head. Instead, knowing about refers to a dynamic 
activity that is distributed across knower and that which is known. 

It is this intersection of individual, context, and activity over time that constitutes the unit 
of analysis when one adopts a situated perspective (Greeno, 1998). The difficulty in finding 
methods for capturing this unit of analysis lies in the fact that it is distributed spatially and 
temporally across these reciprocal components (Barab, Fajen, Kulikowich, & Young, 1996; 
Young et al, 1997). In spite of the challenges in capturing such a dynamic and distributed unit of 
analysis, it is imperative that educators continue to explore innovative methodological 
approaches that capture learning as it emerges within rich environments so as to inform 
instructional practice and design. 

To capture the process of learning in situ, we have been developing an innovative method 
for tracking the emergence, evolution, and diffusion of practices, concepts, and artifacts that 
occur across extended time frames (Barab, Hay, Barnett, & Squire, 1998). We have found this 
method particularly useful in carrying out design research (Brown, 1992), in which we are 
designing entire courses, examining the impact of various interventions on the learning process, 
and feeding this information back into the next iteration of the course. Our methodology is 
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designed to capture occurrences distributed across time and space that influence/constitute a 
learner’s understanding, providing information on how environmental particulars contribute to 
evolving understandings. We believe that a learner’s ultimate understanding of any object, issue, 
concept, process, or practice can be attributed to, and is distributed across, the network that these 
occurrences form. It is in this sense that we see cognition as distributed, embodied, and situated, 
and it is with the goal of capturing knowing-in-the-making that we advance our Constructing 
Networks of Activity methodology. 

UNDERSTANDING AS PRACTICE 

Since the cognitive revolution of the 60’ s, representation has been the central concept of 
cognitive theory and the representational theory of mind has been the most common view in 
cognitive science (Gardner, 1985; Fodor, 1980; Vera & Simon, 1993). The central tenet of this 
position is that “knowledge is constituted of symbolic mental representations, and cognitive 
activity consists of the manipulation of the symbols in these representations, that is, of 
computations” (Shannon, 1988; p. 70). In contrast, others have argued for a shift away from a 
representational theory of mind toward a theory of practice and participation in which cognition 
is considered situated and distributed across time and space (see Greeno, 1998; Kirshner & 
Whitson, 1997; Salomon, 1993). A move toward activity-based (participatory) theories and 
away from a representational epistemology is also consistent with the works of Dewey 
(1925/1981), and Quine (1969), who rejected conceptual representations as primary in 
explaining knowledge. Instead, they viewed learning as taking part or being able to participate, 
and this process involved whole persons (not isolated minds) and rich contexts (including other 
individuals) (Barab & Duffy, in press). 

From this perspective, “knowledge,” perhaps more aptly termed “knowing about,” refers 
to a process distributed across the knower, the environment in which knowing occurs, and the 
activity in which the learner is participating — a dynamic unfolding cycle of perception-action 
(Barab, Cherkes-Julkowski, et al., in press). Knowing and context are inextricably linked 
(Greeno, 1998). Learning is thus (re)conceived as fundamentally connected with and 
constitutive of the environmental particulars in which it is nested (Cobb & Yackel, 1996; Lave, 
1997). From this perspective, it is the complex and dynamic intersection of individual, context 
and activity over time that constitutes the unit of analysis (Engestrom, 1993; Greeno, 1998; 
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Lave, 1988). How much simpler it was to accept cognitive science's isolated problem solver as 
the unit of analysis, wherein observations in laboratory settings serve to "analyze ... structures of 
the informational contexts of activity, but [with] little to say about the mutual interactions that 
people have with each other and with the material and technological resources of their 
environments" (Greeno, 1998, p. 6). However, from a situative perspective, assessments and 
methodological approaches that focus solely on the individual learner are necessarily limited, 
and will fail to provide the rich contextual descriptions of knowing about that are so fundamental 
to situativity conceptions of cognition. 

ACTOR-NETWORK THEORY 

For the most part, situative theories have been developed and researched through 
anthropological studies, frequently with a focus on activity that takes place in contexts of daily 
living and working (Engestrom, 1993; Knorr-Cetina, 1981, 1992; Lave, 1991; Hutchins, 1993; 
Latour & Woolgar, 1979; Lave & Wenger, 1991; Rogoff & Lave, 1984; Saxe, 1992; Scribner & 
Cole, 1973; Suchman, 1987). The central focus of these studies has been on understanding 
"context" as a social world constituted in relation with acting persons (see Lave, 1988 for a 
collection of chapters related to this issue). For our research, we chose an approach to 
conceptualizing and capturing these relations that builds on Actor-Network Theory (ANT). The 
ANT is a sociological approach developed by Callon and Latour (Callon, 1987; Callon & Latour, 
1981; Latour, 1987) to trace the emergence, evolution, and diffusion of scientific knowledge and 
artifacts across a society. "This approach allows researchers to position actors within a larger 
context and reflect on their specific 'mediating' roles and to formulate appropriate practices of 
intermediation" (Gartner & Wagner, 1996, p. 187). 

For actor-network approaches, organizations, communities, and even technical artifacts 
such as published articles are constituted of (and interrelations are fused through) interlocking 
networks of actors. Actors, generally speaking, can be human and non-human, collectives or 
individuals, material or non-material, with each individual, group, company, technology, belief, 
raw material, or artifacts constituting an actor (Roth, 1996). They are referred to as actors 
because in some way they act on the historical development of the phenomenon (whether it is 
physical, social, or conceptual in nature) being studied. A particular actor-network consists of 
heterogeneous, but seamless, interactions among actors, with no a priori distinction among 
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human and non-human actors; they constitute "symmetrical anthropologies" (Latour, 1993). In 
addition to being a part of networks, actors are also constituted by networks; that is, they are both 
constitutive of and constituted by networks of actors, reciprocally determining and being 
determined by the interactions in which they are a part. 

The network approach thus allows us to eschew the reductionism of more traditional 
approaches that focused a priori on the individual or the social, the social or 
technological, content or context, human or nonhuman actors; it integrates all these 
dichotomies)... Understandings that arise out of such analyses are heterogeneous because 
they focus on more than just the technical or the social, content or context, and human or 
non-human by stressing the coevolution of each of these pairs. (Roth, 1996, p. 183). 

Such an approach is particularly appropriate for researchers who recognize that cognition is 
distributed across the task, the individual, and the (physical and social) setting (Garfmkel, 1967; 
Kirshner & Whitson, 1997; Lave, 1988; Lave & Wenger, 1991; Roth, 1996). Further, it 
complements the anthropological tradition of ethnomethodology, which is aimed toward 
providing "grounded" accounts of social action through understanding the context in which 
practice takes place — in spatial, cultural, and social terms (Garfinkel, 1987). 

The ANT has been applied by anthropologists to understand the historical development 
of computer technologies and paper-based manuscripts (Gallon, 1986; Latour, 1987), by 
sociologists to understand screening programs and the historical development of medical 
technologies (Prout, 1996; Singelton & Michael, 1993), by computer scientists to understand 
systems design (Gartner & Wagner, 1996), by various authors to forward ontological 
commitments (Lee & Brown, 1994; Radder, 1992), and by educational psychologists to 
understand learning in open-ended learning contexts (Barab, Hay, Barnett, & Squire, 1998; Roth, 
1996). In many of these applications, actor-network analysis was applied not just as a 
sociological exercise but as an analytical tool (Gartner & Wagner, 1996; Prout, 1996; Singelton 
& Michael, 1993).' For example, Gartner and Wagner (1996), in understanding systems design, 
actually built actor networks of two case studies of companies in Germany and Austria, focusing 
on the various ways actors and intermediaries contribute to the work and to systems design, how 
legitimate agenda are created, and the relations between systems design and these other agenda. 
By mapping out these evolving networks, including actors, artifacts, procedures, and 




u' u u . 7 



Constructing Networks Of Activity 5 



intermediary links they were able to understand the mediations and mediating influence of 
artifacts, cultures, and political agendas with respect to systems design. 

Wolf-Michael Roth (1996) used actor-network theory to examine how learning unfolds 
within student-centered classrooms. Roth (1996) investigated the way resources (i.e., any piece 
of information, objects, tools, or machines) and practices (i.e., embodied tool-related laboratory 
skills and understanding and application of concepts) influence a classroom community. He 
used actor network theory to portray the diffusion of resources and practices within the context 
of science classes, so as to provide empirical evidence for understanding the distributed and 
situated nature of learning and knowing in school settings. Central to this research was the 
notion of tracers . Tracers can refer to facts, practices (tool and concept related), student 
productions (e.g., projects developed), or understandings that can be observed and followed over 
time (Newman, Griffin, & Cole, 1989; Roth & Roychoudhury, 1993). In Roth's (1996) research, 
tracers are selected and then their history is followed through the network. 

For example, in one study, students participating in a 13-week long unit on civil 
engineering were expected to develop a bridge that had to have a minimum span of 30 cm using 
toothpicks (Roth, 1996). Analysis of videotaped classroom interactions and field notes, allowed 
him to trace the diffusion of students’ adoption and understanding of resources (facts, objects) 
and tool-and concept-related practices. Results suggested that the process of learning a tool- 
related practice (in this case, the use of glue guns to connect toothpicks) actually transformed the 
community, as the children began to embody the practice. Roth was able to document the 
trajectory from peripheral participant to core participant, a process that occurred in relation to the 
implementation of particular resources and practices. Also of interest was that specific artifacts 
(e. g., placing a flag on one’s bridge) tended to spread relatively easily, and had little effect on 
the overall composition of the community, while tool-related practices diffused more slowly and 
had the greatest impact on the overall composition of the community. In contrast to both of these 
was the diffusion of intellectual- or concept-related practices (for example, the notion of 
triangulation), which was extremely slow and only occurred with constant prompting of the 
teacher. On the other hand, students were able to express the fact (triangulating supports 
structures) rather quickly. 

Consistent with ANT, no one actor, whether it be the computer, teacher, other resources, 
or student is given a priori priority over others in explaining the development of the tracer. "Any 
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aspect of a collective or technology can be understood by following one or more actors of the 
network that constitutes the collective or technology" (Roth, 1996, p. 186). Further, many of the 
actors can have “simultaneous multiple membership” in histories associated with multiple 
tracers. Examination of these networks and the nested tracers can provide insights into the 
possible roles of teachers and how students construct meanings in participatory learning 
environments. 



DEVELOPING A METHODOLOGY 

Drawing primarily on the leamer-as-processor metaphor, there is a long tradition of 
methodological practices for assessing the learning process (e.g.. Sax, 1989; Wittrock & Baker, 
1991. Because the assumptions to this approach start with the student as unit of analysis, 
"attempts are made to control rather than measure the information within the testing environment 
and questions are rarely asked regarding the interaction between the agent and environment at 
the time of assessment" (Young et al., 1997, p. 135). Given the individual or, more specifically, 
the mind of the individual as unit of analysis these traditional methods appear to deal more or 
less adequately with capturing the phenomena of paradigmatic interest (Brown, 1992; 

Schoenfeld, 1992). However, as one moves to a situativity perspective it is the complex and 
dynamic intersection of individual, environment, and activity over time that constitutes the unit 
of analysis (Greeno, 1998; Lave, 1993; Engestrbm, 1993). That is, the focus is on learning and 
knowing in practice, not on some hypothesized reification of practice. 

The adoption of situativity perspectives, which take seriously the dialectic relations 
among individual, environment, and practice, suggest a need for new methodological practices 

9 

that are more able to capture these dynamic relations. This state of affairs was described by 
Schoenfeld (1992): 

Since becoming a student of cognition, I have been in the awkward position of having at 
my disposal a wide range of scientific methods and perspectives . . . alongside the 
recognition that they are frequently inadequate to deal with the issues I find central. Each 
established method provides some insights, but those insights (whether from 
ethnographic analyses or cognitive modeling, to pick two paradigmatic examples) have 
been, from my perspective, either too narrow or too partial to give a true sense of what 
thinking and learning are all about, (p. 180) 
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The design of new methods to capture one's phenomena of interest is a risky undertaking, and the 
"the pursuit of what one finds 'challenging and interesting' can lead to an ad hoc empiricism that 
is theoretically vacuous" (Schoenfeld, 1992, p. 181). Therefore, in presenting this methodology, 
we have attempted to address the following criteria, which is consistent with those criteria 
advanced by Schoenfeld (1992). 

1. Establish the context, describing the issues to be addressed. 

2. Describe the rationale for the method. 

3. Describe the method in sufficient detail that readers who wish to can apply the 
method. 

4. Provide a body of data that is large enough to allow readers to (a) analyze it on their 
own terms, to see if their sense of what happened in it agrees with the author's, and 
(b) employ the author's method and see if it produces the author's analyses. 

5. Offer a methodological discussion that specifies the scope and limitations of the 
method, as well as the circumstances in which it can profitably be used, and that 
treats issues of credibility and trustworthiness. 

In the previous sections, we focused our discussion around the first two criteria, with 
continued discussion of these throughout the paper. In the next section, we will describe our 
methodology for tracing the emergence, evolution, and diffusion of knowledge and artifacts 
across groups, providing examples of data and analyses. Although we do not view it as possible 
to provide a body of data large eough to truly situate the reader, we do provide examples and 
analysis that are rich enough to illuminate for the reader the process and potential of this 
methodology for capturing knowing-in-the-making. We will then close with a discussion of the 
scope and limitations of this manuscript, including issues of reliability and validity. 

CONSTRUCTING NETWORKS OF ACTIVITY METHODOLOGY 

The Constructing Networks of Activity (CNA) methodology discussed in this paper 
operationalizes ANT into an analytical tool for capturing the distributed and situated nature of 
learning environments — our focus is on what Bereiter and Scardamalia, 1993, referred to as 
intentional learning environments. The CNA methodology allows researchers to identify 
relevant data from a complex, evolving environment, and then to organize it into a web of 
meaning that can illuminate the emergence and historical development of various practices. 
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concepts, and resources occurring over extended time frames, as well as the potential of a 
particular environment for supporting these processes. The network is the central organizing 
metaphor for both the theoretical ANT framework and the CNA methodology. The key elements 
of any network are nodes and links. 

In the analytical tools of Event-State and Causal Networks (Miles & Huberman, 1984), 
the nodes are defined as time-dependent events that “happen” (a meeting, a conversation, or a 
mouse click) or they are defined as a state of mind (student frustration or pressure by parents). 

In CNA, nodes are analogous to what qualitative researchers describe as “units” or “chunks of 
meaning” (Lincoln & Cuba, 1985). These discrete “chunks of meaning” must be identified 
within the continuous flow of data that comes from an authentic learning environment. In the 
CNA methodology, we call these units Activity Nodes. Lincoln and Cuba (1985) described two 
criteria for units of analysis: they must be heuristic and they must be the smallest piece of 
information about something that can stand by itself. 

Beginning with the first criterion, heuristically. Activity Nodes, minimally, contain 
information about the material, conceptual, or social object of focus, who the initiators are, who 
the participants are, what practices the initiators are engaged in, and what resources are being 
used. Specifically, the critical categories of the Activity Node are the Issue at Hand, the 
Initiators, the Participants, the Resources, and the Practices (see Table 1). Each of these larger 
categories may also have subcategories , for example, under practices we have additionally 
delineated between those that were instructor-related, student-related, tool-related, and modeling- 
related practices. Each of the five broad categories and any subcategories then contain specific 
codes , for example, under instructional practices we included coaching, Socratic questioning, 
lecturing, just-in-time lecturing, among others. 

[insert Table 1 about here] 



Defining the Central Catagories 

The Issue at Hand or theme is the “direct” object of discussion or manipulation. It can 
refer to a resource, a practice, or a conceptual tool/process. For example, an Activity Node in 
which there is a discussion between two students about the size of the moon is a straightforward 
example of an Issue at Hand. The Issue at Hand is the object of discussion, the Moon. When a 
student is animating a model of the Moon, the Issue at Hand is the object of manipulation, the 
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Moon. It is important that the action not be confused with the direct object, even though the 
direct object can be an action. In the previous example, animating was the action, not the direct 
object of discussion or manipulation. However, when discussing a bug in an animation of the 
Moon, there are two Issues at Hand: the object. Moon, and the action, animating. This is because 
the action of the Activity Node is the “discussion” and the direct object of that discussion is both 
“animation” and the “Moon.”^ The Issue at Hand serves a primary identification and labeling 
function in the overall-coding scheme. 

An Initiator is an individual, technology, object, or group (when engaged in a practice as 
a single unit) that is producing an action. With respect to our research interests. Initiators are 
important to focus on for two reasons. First, because we study learner-centered environments in 
which the teacher is no longer the primary initiator, how and when learners initiate their own 
learning is a crucial issue. Second, because students are working with interactive models, new 
activities are often initiated by the running models themselves. That is, after creating a model 
and running it, the model stimulates new activities and discussions for learners in which is 
otherwise difficult to engage. Although we listed only observable initiators, it is important to 
note that activities do not emerge in a vacuum; rather, they exist within a context that is 
reciprocally constituted by the cultural surround. In this fashion, the cultural surrounding could 
arguably be considered an Initiator involved in defining the specifics of the Issue at Hand or even 
which practices emerge. However, it becomes impossible, and we believe, overly presumptuous, 
to define the numerous aspects of cultural influence that could possibly be considered Initiators. 
Therefore, we have not included these non-observable, yet potentially important, cultural 
influences in our coding scheme and must acknowledge this as a limitation. The third category. 
Participants, are all other individuals who are involved in an Activity Node but not initiating the 
action. 

The fourth category we identified as a critical element of learning is the Resource. A 
Resource is “any piece of information, object, tool, or machine” that an Initiator uses to carry out 
a Practice (Roth, 1996, p. 191). These include student-developed artifacts. In addition to 
technological tools, our definition of tool includes those of a conceptual nature (i.e., relative 
scale as a way of perceiving a model). We contrast a "resource" that is in use, to an "artifact" 
that is simply available. An artifact is transformed to a Resource when an Initiator as part of a 
practice uses it. 
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The final critical element of any Activity Node is the activity itself. The Practice is an 
activity carried out by an Initiator or Participant who is using a resource. There can be many 
different categories of Practice within a Network of Activity. Some of these categories of 
Practice will be associated with specific types of initiators (i.e., student or instructor practices), 
some will be associated with different content areas (i.e., mathematics, science, or English 
practices), and some associated with particular resources (i.e., WWW, word processing, or ruler 
practices). In fact, there are often several different Practices simultaneously conducted within 
one Activity Node. For example, in the work of Roth (1996) discussed above, two students 
collaboratively building a toothpick bridge with a glue-gun might be involved in three Practices. 
They are engaging in the learning practice of discussion, the tool-related practice of using the 
glue-gun, and the mathematics/engineering practice of triangulation all within one Activity 
Node. However, each Practice involves a specific resource: prior experiences that the students 
are drawing into their discussion in the case of the learning practice, the glue-gun and the 
toothpicks in the case of the tool-related practice, and the toothpicks in the case of the 
mathematics/engineering practice. In review, each Activity Node has, minimally, an Issue at 
Hand, the Initiator, the Participant, the Resource, and the Practice. We will now turn to the 
operational consideration of creating a Network of Activity. 

Coding Nodes 

Operationally, the identification or “chunking” the raw data into units or Activity Nodes 
is the first step in the creation of the network. Activity Nodes are identified as an activity 
occurrence that are judged to be a “significant happening” in the learning context, and are 
delimited by a change in theme, practice, or incidence. What qualifies as a significant happening 
is somewhat subjective and specific to the needs and interests of each particular research context. 
This subjective judgement goes to Lincoln and Cuba’s (1995) second criterion for a unit of 
anlaysis, “smallest piece of information.” For example, one researcher might be interested in 
capturing fine-grained practices (e.g., mouse clicks or turn-taking in a conversation) while 
another researcher may be interested in more molar units (e.g., moving an object across a screen 
or a planning discussion). 

Once the Activity Nodes are identified, the next step in our methodology involves 
developing codes for the critical categories (e.g.. Practices, Resources, Initiators) and for any 
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relevant subcategories (instructional-related, tool-related, modeling-related practices). In our 
research, the subcategories are usually based on content analysis and pilot research. However, 
the sets of specific codes are then developed through weekly meetings in which members share 
field notes to develop grounded categories of codes (Glaser & Strauss, 1967). In other words, 
these codes are not applied top-down, but emerge from the data. In this fashion, the 
development of the coding types is an evolving and iterative process that takes extended group 
discussions in which the evolving codes and subcategories are continually tested against 
empirical data. Based on these discussions, a computerized coding form is then developed with 
a relational database that allows researchers to input the basic information (time, date, coder, 
tape, etc.), a written description, a rating of the conceptual richness, and the codes with respect to 
the Issue at Hand, Initiators, Participants, Resources, and Practices for each Activity Node (see 
Figure 1 for an example of a coding form of our Virtual Reality Astronomy course). 

[Insert Figure 1 about here] 



Developing Links 

The second main feature of a network is the links that connect the nodes. We can 
conceptualize the links as anything that ties one Activity Node to any other Activity Node. Thus, 
conceptually, all our codes serve as links between Activity Nodes. That is, time links nodes 
historically, practices link nodes of similar practices together, resources link nodes of specific 
resources used together, and Initiator and Participant codes link nodes of people and technology. 
These linkages through all the nodes of a given database can be envisioned as akin to a densely- 
woven, highly complex "knot" of nodes and links. This knot would not be particularly useful 
without the ability to tease out issues and stretch the knot in theoretically interesting ways. The 
question becomes. What links are productive to graphically represent within a network to address 
the underlying questions? In Event-State and Causal Networks (Miles & Huberman, 1984), 
particular types of links (causal links) are selected for representation to demonstrate the policy 
events that have led up to particular district level outcomes. In our work, to bring order to this 
knot, we have developed a method to visualize this database in a fashion that enables us to 
explore the issues around the emergence, evolution, and diffusion of practices, concepts, and 
resources occurring over extended time frames. We have established two types of links: Primary 
Secondary. The Primary Links are focused on people as they move from one Activity Node to 
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another. For example, if Patricia were a part of two Activity Nodes, she would be a Primary 
Link. If she had a collaborator (say Tim) who participated in the two Activity Nodes, then he too 
would be a Primary Link between the two Activity Nodes. However, if Tim were not a part of 
the second Activity Node but were a part of a third node, then he would link the first node to the 
third node. This is important conceptually because it is our contention that the study of the 
emergence, evolution, and diffusion of various practices, concepts, and resources occurring over 
extended time frames can only be traced through the activities of people. 

Secondary Links conversely, are not represented through lines connecting nodes (as are 
Primary Links), but rather are represented with node indicators (color and patterns) on the 
Activity Nodes and can be traced through the Primary Links. These Secondary Links can be any 
of the codes within the coding scheme, but we have focused on the link across the following 
codes: concepts, practices, resources, and student-developed artifacts. 

Visualizing Nodes and Links 

Operationally, the visualization and representation of links starts with the visualization of 
the nodes on a time-line. This process begins by developing graphs in which the Y-axis 
represents time and the X-axis represents student Initiators. In this discussion, we use an 
example of one group of four students. However, the CNA methodology can be used with larger 
groups. The Activity Nodes are represented by bars that indicate actual time duration for each 
node organized by initiator (see Figure 2). The time intervals on the y-axis are defined by the 
unit of analysis of interest to the researcher. In this case, they represent one-minute blocks. 

[Insert Figure 2 about here] 

These node bars are then abstracted into circles that represent relative position (see 
Figure 3). We acknowledge that these ovals do not have a one-to-one mapping in terms of actual 
node duration or in terms of when they occurred. However, for the purpose of this analysis, the 
graph does maintain the relative position of nodes with respect to the time when they occurred. 
For example, the two nodes in which Jose and Mary initiated at time 1 had different duration but 
occurred during the same time period relative to the nodes displayed for time 2. In some cases, 
the actual node overlaps the time periods (e.g., the node initiated by Susie occurs across times 3 
and 4). As a general rule, we line up nodes with similar start times. In this case, we chose to 
display the node at time 3 on Figure 2, but a reasonable argument could be made for displaying it 
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at time 4 instead. However, for our purposes, the placement of a node at time 3 or 4 would not 
change the overall picture in a meaningful way; that is, this placement would not effect how we 
trace and interpret the historical development of a particular practice, concept, resource, or 
student-produced artifact. 

[Insert Figure 3 about here] 

Lastly, links are added to represent the participants of a node (see Figure 4). This 
represents a person’s path though the Network of Activity. The final goal of building a network 
is to trace the historical development and diffusion of the topic of interest (Issue at Hand, 
Practice, or Resource). Once the researcher has constructed a complete Network of Activity, the 
next step involves selecting those nodes related to the topic of interest. Figure 5 represents a 
“tracer network,” referring to a network that signifies the tracing of a particular Issue at Hand, 
Practice, or Resource. This is simply a hypothetical example to illuminate a section of a 
network. Below we will present actual examples from our research context. 

[Insert Figure 4 about here] 

In this hypothetical network, all the dark nodes in Figure 5 represent those nodes of a 
particular Practice. In this hypothetical vignette, Mary is working by herself on “the practice” in 
the first two time periods, and the other student, are working in a group initiated by Jose on 
something else. At time 2, Jose calls Mary over to show him and Tom how to do the practice. 
Then, Tom goes back to his computer and does one other thing before he asks Jose to help him. 
Susie tags along. Susie returns to her computer to work on the Practice. At this time, Mary and 
Jose have returned to their computers and begun something else. 

[Insert Figure 5 about here] 

In the above discussion, we described the process of constructing networks. In our 
research, the process from observation to analysis has involved the following steps: 

1) Devising a means for capturing interactions — we have found it useful to use video; 

2) Reviewing the captured interactions, “chunking” them into discernible units of 
analysis that we have described as Activity Nodes; 

3) Recording information related to the specifics that constitute each Activity Node 
(see Figure 1); 




16 



Constructing Networks Of Activity 14 



4) Developing a visual representations of the Network from time bar representations, 
to abstracted nodes representation, and linking these nodes together to form a 
complete network (see Figures 2-4); 

5) Selecting the particular Issue at Hand, Practice, or Resource to serve as the tracer; 
and 

6) Tracing the historical development of the particular tracer over time by graphing 
out all the related nodes and examining the path (see Figure 5). 

INSTANTIATING THE CATEGORIES (OUR RESEARCH) 

To illuminate the use of the CNA methodology within a classroom, we will now describe 
one of our particular research contexts and how we have used CNA to trace the historical 
development of various concepts, practices, and resources. In particular, we will demonstrate 
how we have used the CNA methodology to capture the emergence, evolution, and diffusion of 
concepts, practices, resources, and objects in our Virtual Solar System (VSS) project (Barab, 
Hay, & Barnett, 1999; Hay, Johnson, Barab, & Barnett, in press). However, this is an illustrative 
example of applying the CNA methodology to a particular context and the reader should not 
view this one instantiation as representative of the myriad of potential uses for this tool. 

Research Context. 

In this research, we have been exploring leaming/instruction within collaborative, 
technology-rich, project-based learning environments (Barab, Hay, & Duffy, 1998). The VSS 
Project is an experimental undergraduate Astronomy course taught at a midwestern university. 
For this project, we completely transformed this traditional lecture-based course into a project- 
based course (Barab, Hay, & Barnett, 1999; Hay, Johnson, Barab, & Barnett, in press). Where 
previously listening to lectures constituted the primary learning activity, in the VSS course, 
listening to lectures was replaced by students’ building three VR models of different aspects of 
the Solar System: 1. the Earth, Moon, and Sun, 2. the entire Solar System, and 3. a learner- 
defined object of the Solar System (a comet, the asteroid belt, etc.). Students worked in teams of 
two to four members, using high-end graphics computers with a direct manipulation VR creation 
software that allowed the projects to be exported directly to the World Wide Web (see Barab, 
Hay, & Barnett for a full discussion of this course). 
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We used video cameras to capture each of four student groups as they constructed their 
VR worlds within a computer lab in which there was a one-to-one student-computer ratio. 
Although we had a separate video camera directed toward each group, it was not possible to 
capture all of the information on each of the computer screens along with student gestures and 
dialogue. Therefore, a researcher was assigned to each group during class time, allowing for 
much of the coding (selecting interactions and filling out the nodes) to occur “on the fly.” 
Videotapes were later inspected to ensure accuracy of nodes. To increase consistency across 
group coding and to provide all researchers exposure to each group, we rotated groups on 
different days and performed member checks by having researchers examine each others’ coding 
of nodes. 

Coding Examples 

The first step in adapting the CNA methodology to a particular research context is to 
choose the subcategories and the respective codes. We have found it most efficient to enter these 
subcategories into a database program in which the items can be accessed as pop-up menus (see 
Figure 1 for the coding form developed on our VSS context). In addition, it is essential that 
these items be editable and that items can be added as new Issues at Hand, Practices, Initiators, 
Participants, and Resources are identified. Included in our form are fields for the group number, 
date, start and stop times, the name of the person coding the form (coder), the Issue at Hand 
(practices, concepts, and objects). Conceptual Richness, Initiators (objects, students, mentors). 
Practices (tool-related practices, modeling related, mathematical, group project, instructor, 
student). Context, Resources (including concepts used as tools such as logarithms), participants 
(student, mentor). Tracers, and a Description of the Node. The five major category labels were 
discussed above (see Table 1); however, in making the CNA methodology a useful analytical 
tool, we have found it necessary to include additional fields to our form. We will briefly explain 
these additional categories, which include group, date, start and stop, conceptual richness, 
conceptual tools, and description fields. 

The group field is useful for searching and sorting the records when using the CNA 
method in a context with more than one group working on similar projects and, potentially, being 
combined in the same database. The date and start and stop times allow researchers to sort 
records and follow the historical development of various items. Conceptual richness is a rating 
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from “0” to “9,” with 9 representing nodes involving interconnections of ideas or systems-level 
understandings of the particular domain of interest. For example, the concept of "line of nodes" 
is central to the domain of astronomy. It refers to the imaginary line formed by where the plane 
of the Moon's orbit intersects the plane of the Earth's orbit. An eclipse is possible only when the 
Moon, Earth, and Sun are all on the line of nodes. An Activity Node would be judged a 9 when 
it involves this systems-level appreciation of these nested concepts. 

Concepts as resources refers to concepts when they are being used as tools to support the 
carrying out of a practice (e.g., using logarithms to determine scale sizes). Here we are making 
the same distinction as we made with other types of tools, that is, we need to code them 
differently where they are being used versus when they are being talked about. If students were 
talking about what logarithms are, then the logarithms would be the Issue at Hand. However, if 
they were using logarithms as a resource to determine planet distances in a mathematical way, 
then they are using logarithms as a resource. 

It has also proven necessary to add a field to our database for brief textual descriptions 
that allow researchers to gain a rich contextual picture when examining nodes constructed at 
other times and by other researchers. In addition to textual descriptions, our database forms 
include a field for displaying the segment of video associated with the node. 

Developing Codes 

The codes related to our categories and subcategories emerged through the process of 
grounded theory development, in which our data and emergent interpretations interacted in a 
dialectic fashion, reciprocally informing and being informed by the other (Glaser & Struass, 
1967). In addition, we performed content analysis and examined the literature to enhance our 
"theoretical sensitivity" (Schatzman & Strauss, 1973), engaging in a dialogue between previous 
theory and current data. The resources were simply lists of resources that were available to 
learners in the environment. However, over time, new resources (physical and conceptual) 
became available, and additional types were added to the resource field. All practices from all 
categories, in addition to being available in the Practices category, were also made available 
under Issue at Hand, as were Resources. Lastly, all students and instructors were added to 
Initiator and Participant categories, and group, date, and coder categories were also added to the 
database. 
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Coding Scenarios as Activity Nodes 

In this section, we present a scenario from our research program to demonstrate how the 
CNA methodology has been used to develop rich interpretations of these experiences. We have 
included four scenarios in this paper to support researchers in applying this approach. However, 
based on the suggestions of the reviewers, we have included scenarios 2 through 4 as appendices 
but will refer to all four in this set-up. These scenarios are presented as four different examples, 
each of which includes multiple interactions coded as multiple Activity Nodes. The first 
scenario involves the coding of a practice and how it changes from an Issue at Hand to a tool- 
related Practice over time. The second scenario illustrates how the line of nodes changed from a 
concept, to an object, and finally to a conceptual tool over time. The third scenario illustrates the 
coding of a conceptual tool (e.g., logarithms) and how this tool becomes black boxed into a 
student-created inscription where it can be used as a table resource by other students. Black- 
boxed describes the process by which a piece of machinery or a set of commands becomes 
compiled. At this point, the commands are no longer the focus of the activity, but can be used, 
transparently, to perform some other activity. The fourth scenario illustrates how project 
constraints become instantiated into the model, and how the CNA methodology allows 
researchers to capture the rich history of interactions within which these interactions are built. 

In these examples, we will illustrate how we have parsed a scenario into separate Activity 
Nodes, and how to code each one of the Activity Nodes. In each scenario, we will begin with a 
set-up that will contextualize the reader to the goals, motivations, and the activities prior to the 
scenario. Then we will present the collected data from the scenario broken up into turn-taking 
segments. The far right column indicates how we chunked the data into Activity Nodes. Last, 
each Activity Node will be coded using multiple category labels. Students include Marvin, 

Keith, and Roger, and instructor is Igor. Following the dialogue, we then list the particular codes 
our research team selected for this dialogue (see Table 2). 

Scenario 1 I Viewpoints') : The following set of interactions illustrates, how setting viewpoints 
moved from an Issue at Hand to a tool-related Practice over time. Viewpoints refer to 
perspectives of "camera positions" that can be placed in a virtual reality model, allowing viewers 
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of the model to immediately shift to various locations. The interaction begins after Marvin has 
built an Earth-Moon-Sun system and animated all the pieces. 



Participant 

Marvin 

Marvin 

Keith 

Marvin 



Keith 

Marvin 

Keith 

The dialogue 

Participant 

Marvin 

Marvin 

Igor 

Marvin 

Roger 

Igor 

Marvin 



Description 

[Marvin is setting a viewpoint so that end-viewers can go to a 
perspective on the VR world that he determines.] 



Node 



Okay, in a lunar eclipse does the Earth block the light of the Sun? 

Yeah, let's see [then Keith moves next to him and says "show me " as he 
looks at the computer] 

So, in a lunar eclipse, wouldn’t it make sense to have the camera going 
from the Earth to the Sun [pointing to the screen] ... If we put the 
camera on the Sun and make it face towards the Earth so we can see what 
the Earth is doing when it gets in the way of the light of the Moon. 

[Keith shows Marvin what the eclipse looks like by modeling with his 
hands] 

Oh, so a lunar eclipse is when the Earth blocks the light of the Sun? 

Yes. 



continues. 

Description 



Node 



[Marvin is viewing the virtual model’s viewpoints. He is jumping from one 
to another. He is visibly frustrated] 



I thought I set a viewpoint here yesterday, but it is not working [Marvin's 
inflection hinted at it being a question] 

Okay, so what did you do to set the viewpoint? 

Somehow I put one viewpoint camera right in the center of the Earth, but I 
erased it and now all of the cameras are screwed up. 

We put two cameras up there [referring to the top of the Earth], but I don’t 
understand what happened to them. 

Let’s look for the viewpoints you have now, and delete them and start 
again. 



[They find the two cameras and delete them] 
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Igor 


[Igor does a just-in-time lecture on setting viewpoints so that students can 
demonstrate various aspects of their solar system model with the cameras.] 


The scenario continues: 




Participant 


Description 


N 


Marvin 


[Marvin sets a viewpoint that illustrates a lunar eclipse] 


7 




[insert Table 2 about here] 





Interpreting Data 

We have found it useful to carry out three types of data interpretations on the database of 
nodes generated using the CNA methodology (Barab, Hay, Barnett, & Squire, 1998): as a 
database search tool to support grounded theory development (Glaser & Strauss, 1967); to 
develop Frequency/History graphs, and to develop the full Network of Activity graphs. The first 
use involves using the database of nodes to identify patterns or particular episodes that illuminate 
key characteristics of the phenomenon being studied. This process is useful when characterizing 
course dynamics (Barab, Barnett, Yamagata-Lynch, Squire, & Keating, 1999), or in identifying 
instances related to a particular issue (Barab, Hay, & Barnett, 1999). The second use involves 
examining the frequencies of occurrence of one particular element of a node over a particular 
time frame, for example, nodes of a project as they relate to particular concepts, practices, or 
resources. This representation is particularly useful for getting a broad look at an element, for 
drawing contrasts between groups (e.g., whether two teachers differ in terms of number of times 
Socratic questioning was used), for simply examining the frequency of which a particular 
resource was used (e.g., using the World Wide Web as a resource) or a practice was carried out 
(e.g., how many times scaffolding was used), or to determine the average conceptual richness of 
the coded nodes. The other more exciting data interpretation method is to actually trace the 
emergence, evolution, and diffusion of concepts or practices over time through the entire 
Network of Activity (see Figure 5). It is with this goal in mind that we developed the CNA 
methodology, and it is this function that allows researchers to capture cognition in situ. 

As a Database Search Tool . Barab, Barnett, Yamagata-Lynch, Squire, and Keating 
(1999) used the database to identify important interactions, meaningful patterns, and trace their 
occurrences throughout the semester long VSS course. In analyzing the data for their research, 
they selected various initiators, practices, and concepts and then used the database, fieldnotes. 
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and interviews to identify when and where in the course they occurred. Based on the node 
descriptions in the database and tags in their fieldnotes, they then went to the original videotapes 
and examined complete episodes. Through an examination of the specific episodes and their 
place in the database, they were able to develop an appreciation for the overall course context in 
which a particular episode occurred. Then, using Activity Theory as a theoretical lens 
(Engestrdm, 1987; Leont'ev, 1974), they were able to examine the relations of subject and object 
as mediated by the primary components of an activity system. Zooming back out to the larger 
course context with the database, they were then able to contextualize each episode within the 
course context, and to identify central contradictions that characterized the course activity system 
as a whole. 

Examining the Frequencv/History Graphs . Once the researcher (or, more likely, research 
team) has coded the experience of interest, she can than select various topics to determine their 
frequency. For example, in a summer camp in which we had students building VR worlds, we 
went through the nodes and counted the frequency in which the mentor served as initiator 
(Barab, Hay, Barnett, & Squire, 1998). In this research, mixed initiative interactions (nodes 
where both learner and mentor would initiate questions for example) were coded with both 
leamer(s) and mentors as the initiators; thus student-initiated nodes and mentor-initiated nodes 
are not mutually exclusive. We conducted this analysis by simply adding up the number of 
nodes that were mentor-initiated (first analysis) or that were student-initiated (second analysis). 
With respect to mentors, results suggested that there were less mentor-initiated nodes for Group 
One (107/242 or 44%) than for Group Two (160/238 or 67%). On a related note, there were 
more student- led nodes for Group Two (216/242 or 90%) than for Group One (140/238 or 59%). 
From these results, it became apparent that Group One was more successful in empowering 
learners and more consistent with the learner-driven, participatory framework from which the 
camp was developed than was Group Two. 

In this same research project, we were able to use this strategy to look at group 
differences with respect to whether teachers or students defined tasks and goals. Through an 
examination of these networks, we were able to gain a very different view of the groups, with 
Group Two having more nodes (72 of 242) focused on task definition than did Group One (23 of 
238). Frequency counts with respect to which individuals were the initiators are displayed in 




23 



Constructing Networks Of Activity 21 



Figure 6. We found it easier to represent the data as a frequency chart than as a "tracing" 
network. However, the data could have also been represented in the network form. 

[insert Figure 6 about here] 



Connecting Nodes to Form Tracers . In this section, we trace the historical development 
of students' interactions related to planetary tilt, showing how students operationalized planetary 
tilts in their virtual models of the solar system. First, a textual account of the activity is produced 
and then we interpret these activities as a Network of Activities Graph. Initial database searches 
revealed numerous nodes related to planetary tilts. However, in developing the tracing of tilt, we 
examined these nodes and found clusters (sets of sequential or nearly sequential nodes) in which 
large numbers of nodes related to tilt were present. Examining all of these nodes would take too 
much space, so for this section we focus on a particular cluster of nodes in which students were 
determining how to represent the Earth's tilt in their models. However, we could have also 
chosen other clusters (e.g., those related to the five-degree tilt of the Earth-Moon system). 

The practice of Planetary Tilt involves students using the VR creation tools to tilt a planet 
on its axis. To carry out this Practice, the student needs to understand the notion of Planetary 
Tilt, and then incorporate this into the model. We will now turn to the data to explore how one 
group accomplished this task. 



Participant 

Jose 
Peter & 
Terence 

Peter 

Terence 

Cindi 

Peter 

Jose 



Description Node 

[Jose is texturing his Mercury.] 1 

[Peter and Terence appear confused with respect to whether the Sun has a 2 
tilt and asks Cindi, the instructor. She directs them to a resource and assigns 



them the task of creating a table of all planetary tilts.] 

[Peter takes on the challenge and generates the table.] 3 

[Terence goes back and attempts to use the numeric tools to try to tilt the 4 
Sun.] 

[ Cindi calls the group together and leads a group discussion on what tilt 5 
means.] 

[Peter uses table to describe tilt in the discussion] 6 

[Jose uses a paper model to describe what it means to tilt. Here, tilt is 1 
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becoming transformed from an abstract number into an enacted conceptual 
tool they are using to create a model of the Solar System] 

Cindi [ Cindi summarizes their discussion, however there are lingering doubts 8 

about the direction of the tilt ("clockwise or counterclockwise?’’).] 

Terence [Terence and Jose go to their astronomy book resources to understand 9 

& Jose "which direction you tilt the planet? ’’] 

Terence [After reassuring themselves that they know which direction to tilt the planet, 10 

& Jose they turn to CosmoWorlds, and the PPT. ] 

Terence [As they start to attempt to tilt Earth, they notice that the rotation option is 1 1 

& Jose not available. They have to select the movement option of Relative/Local, 

because Absolute/World rotation does not make sense since a rotation has to 
rotate about something, not everything.] 

Peter [Peter works on rotating the spheres.] 12 

Peter [Peter reports back to the other students that the Earth should be rotated 23 13 

degrees about the Z-axis.] 

Willie [Willie, the technology mentor, confirms Peter’s statement and launches into 14 

a just-in-time lecture about 3D space.] 

Peter [Peter develops the Practice quickly and tilts his planets.] 15 

Terence [Terence and Jose struggle as they work together to tilt their planets.] 16 

& Jose 

Peter [Peter explains, demonstrates, and guides Terence through planet rotation, 17 

placing the correct axis, and the issue of relative vs. absolute.] 

Cindi [Later, Cindi formalizes the reference point as the plane of the ecliptic, the 18 

plane of the solar system that is formed by the Earth’s orbit around the Sun.] 



What can we learn from this interaction through the CNA methodology? On the top 
level, we can see some general trends that we would follow over the life of the project. These 
trends include: 

1. The mentors' interactions with the students were mentor-led whole-group nodes. 
There was only one student-initiated smaller group node that included a mentor 
(Node # 2). 
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2. Peter worked alone more often (Nodes #3, 12, & 15) and Terence and Jose tended to 
work together more often (Nodes #9, 10, 1 1, & 16). 

3. Of the 18 Activity Nodes, 5 nodes involved isolated individuals and 13 nodes 
involved two or more actors. 

As we push deeper into the data and its visualization we can see that Node #15 is the key 
operational node (that is the node when Peter accomplished the goal of tilting the Earth 
correctly) and Node #18 is the key node conceptually where the mentor formalizes the plane they 
were working with as the "plane of the ecliptic." Through an examination of the historical 
context of these two key nodes, the power of CNA can be illustrated. We will do this by first 
generating the Network of Activities Graph for this particular section (see Figure 7). 

[insert Figure 7 about here] 

Examining the nodes that constitute the Network of Activity shows how the general goal 
of tilting a planet evolves through the use of resources and transforms the trivial number of 23 
into a VR model of the Earth tilted at 23 degrees, then to a general understanding of planetary tilt 
and the plane of the ecliptic. First, we see how resources evolve and are shared across people 
and culminate in the operationally-key Node #17 with all the learners and the conceptually key 
Node #18 with all the learners and mentors. 

Next these nodes can be explored to determine how the building of VR models 
transformed the number 23 into a robust scientific concept. In Node # 3, Peter collects and 
reorganizes the textbook information on planetary tilt into a table and then later presents the table 
as a group resource in Node #6. At this point, Peter has not created meaning for these numbers; 
rather, they serve as the foundation out of which future meanings and practices can emerge. It is 
interesting to note that in traditional courses, this would be a point of completion, an answer to 
an end of the chapter question or quiz question. In the group setting (Node # 7), Jose takes the 
first step to embody meaning to the numbers by using a paper model of a planetary tilt, however, 
he is limited because of the medium. As Terence and Jose turn to the computational model, they 
are challenged by the question of direction of the tilt. The computer modeling environment has 
forced them to confront the emerging and still fragile understanding of planetary tilt. In Node # 
9, they use the textbook as a resource to resolve the issue of direction. The model and the 
software resource push them further as they must use the language resources of the software 
(Relative/Local and AbsoluteAVorld) to deepen their understanding of tilt. They accomplished 
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this by first equating tilting with rotating and then confronting the question of "Rotating about 
what?" Peter, working by himself in Node #12, figures out the details, and in Node #13, reports 
on the newly-developed Practice of correctly rotating a planet 23 degrees. In Node #15, Peter 
develops both confidence and proficiency with the Practice and in Node #17 the practice is 
finally appropriated by the entire group. This solidifies the practice of planetary tilts and forms 
the foundation of Node # 18 for which the main concepts of planetary tilt are formalized by the 
mentor. 

Where formal lecture-based courses typically stop is at the individual-centered activities 
demonstrated in Node # 6. The power of the CNA methodology is that it captures and represents 
the situated/distributed cognitions, showing how cognition is contextually-embedded and 
distributed across concrete experiences. This approach allows us to move beyond capturing 
ready-made knowledge (i.e., students parroting back that the Earth has a 23° tilt) to capturing the 
situated dynamics that constitute knowing-in-the-making (i.e., students actualizing this model as 
part of their VR models). 

TRUSTWORTHINESS, USEFULLNESS, LIMITATIONS 
Is the Coding Scheme Trustworthy? 

The original coding scheme was developed by Barab and Hay, with necessary 
modifications occurring while working on actual data with six graduate students. After we 
ironed out many of the details, jointly, on a number of videotapes, individuals coded numerous 
tapes separately. In establishing reliability, two researchers coded the same 60 Min. segment 
separately. Results indicated 88% agreement in terms of number of nodes selected, with one 
researcher selecting 22 interactions to be categorized as nodes and the other selecting 25. 
Examination of the videotapes and the selected nodes suggested that of those 22 nodes, all but 
one corresponded to the same segment in the video. The next step in establishing reliability of 
the coding scheme involved examining the labels selected within each node. On average, both 
coders selected eight categories (pull-down menus) per node (e.g., one Issue at Hand, two 
Initiators, one Participant, two Practices, and two Resources. In terms of the content selected 
within a category, there was 80% agreement. 

Other evidence for reliability was found in the research of Barab, Hay, Barnett, and 
Squire (1998). In this instance, two researchers collaboratively coded 10 hours of videotape for 
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two separate groups building VR worlds. Altogether, 480 nodes were generated, with 238 for 
one group and 242 for the other group. They stated, "given the qualitative nature of determining 
the boundaries of a particular node, the relative consistency regarding the number determined for 
each group to some extent validates our approach at node identification" (p. 17). Although the 
number of nodes derived was similar across contexts, the content of these nodes was clearly 
different (see Figure 6 for an example of group differences in task definition). 

Is the Coding Scheme Useful? 

In terms of use, our goal was to develop a methodological approach that would capture 
the emergence and historical development of learning in situ. More specifically, we sought a 
method for tracking the emergence, evolution, and diffusion of practices, concepts, and resources 
that occur across extended time frames and that are distributed across multiple environmental 
particulars (e. g., computer screens, collaborating individuals, textbooks). To this end, we have 
found the CNA methodology to be useful. It has helped us understand the learning process that 
occurs in open-ended learning environments when conceived from the situativity perspective 
(Barab, Hay, Barnett, & Squire, 1998). In particular, we have been able to compare groups and 
gain insights into pedagogical differences and group dynamics — differences that were consistent 
with interview and observational data. In addition, we have been able to trace the evolution of 
various practices and understandings and to see how environmental particulars (e.g., the 
development of VR worlds, discourse among individuals) contributed and were bound up within 
this evolution. On a related note, this has been central to our design experiments surrounding our 
VSS course, inn which we introduce interventions and use the CNA methodology to examine the 
role these interventions play in the learning process (Barab, Hay, & Barnett, 1999). 

In What Situations is the Coding Scheme Useful? 

In the previous section we alluded to the usefulness of the CNA methodology for 
capturing learning within open-ended environments. Specifically, we examined its utility for 
tracking the emergence and historical development of knowledge within contexts in which 
individuals have access to various resources and learning that is occurring over extended time 
frames. This suggests that the method's usefulness is better leveraged to learning contexts in 
which there is ample opportunity for engagement with various resources and collaborating 
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individuals. Further, evolution of a particular practice or understanding requires environments 
that afford repeated opportunity to participate in the development and application of a practice, 
or opportunity to gain new insights into an understanding. However, this is more than a 
"crisscrossing of the same landscape" — to build on Spiro and Jheng's (1990) metaphor. For the 
evolution of knowledge to take place, there needs to be novel contexts, in a sense, new 
landscapes that support the learner in continual re(negotiation) of practices and understandings. 
We have been using the CNA methodology in collaborative, project-based environments where 
individual members and, potentially, individual groups are focused on shared tasks. Within these 
contexts, there exists a rich opportunity for knowledge diffusion. This methodology will provide 
less insight into traditional didactic lecture environments in which the goal is to transfer specific 
content from the all-knowing teacher to the individual learner with little translation of the 
knowledge under question. This is because there is less opportunity to participate in the 
emergence, evolution, and diffusion of concepts, practices, and resources. 

What are the Limitations? 

With respect to the limitations of the CNA methodology, we have already alluded to the 
need for rich learning contexts in which there is ample opportunity for the historical development 
of knowledge, thus requiring that knowledge development takes place over extended time frames 
and across multiple environmental particulars. Therefore, in order to actually capture these 
occurrences, it is necessary that researchers have large amounts of video data regarding student- 
student, student-teacher, student-tools, and student-resource interactions. Both capturing and 
analyzing this data are extremely labor-intensive, and, in many situations, simply an impractical 
task. Therefore, this approach is appropriate for researchers but has little application, at this 
stage, to the classroom teacher. 

Other limitations include the time-consuming process of training coders, the qualitative 
nature of determining the boundaries of a particular node, and the requirement that one must be 
able to capture a significant amount of the relevant interactions. Within the summer camp 
context researched by Barab, Hay, Barnett, and Squire (1998), they found it tenable to 
hypothesize that most of the meaningful interactions related to the particular tracers of interest 
happened under the lens of our video cameras. However, in the context of a university course 
that happened over 15 weeks, there were significant learning occurrences that happened outside 
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of the formal educational environment being studied. Therefore, it is unreasonable to believe 
that we have captured all of the meaningful interactions. In fact, there were many occasions in 
which students gathered outside class or on weekends out of view of our video cameras. 

There have been numerous critiques targeted at ANT more generally. For example, 
Kaptelinin (1996, p. 46) stated, "the relations between agents and tools cannot be symmetrical," 
whereas the power of ANT is that is does treat these as symmetrical, serving a useful role in the 
CNA methodology. Engestrbm & Escalante (1996, p. 344) had other criticisms of ANT: "In its 
search for convergence, irreversibilization, and closure, this kind of analysis overlooks the inner 
dynamics and contradictions of the activities of the various actors in the network." He further 
stated that "the concepts of trust and reciprocity, so central in new theorizing on network 
organizations, and the whole contradictory dialectic of cooperation and competition, are 
curiously missing in the vocabulary of actor-network theory" (p. 46). 

In response to these criticisms, it is important to reaffirm that we are simply using ANT 
as a structural framework for the development of our analytical approach and not as a theoretical 
framework for conceiving the relationship among human and non-human actors. In this manner, 
our methodology is useful for identifying those interactions related to a particular tracer of 
interest. It is then the responsibility of the researcher to contextualize these interactions in terms 
of the larger context in which they unfold. For example, Barab, Barnett, Yamagata-Lynch, 
Squire, & Keating (1999) used the CNA methodology to identify the frequency of occurrences 
related to a particular tracer, and then used Activity Theory (Engestrbm, 1987) to contextualize 
these in their larger context (activity system). 

CONCLUSION 

All too often, assessments capture the products of learning as conceived from a 
representational perspective in which the individual's mind becomes the unit of analysis. 

Because the individual is the unit of analysis, questions are rarely asked regarding the interaction 
between the agent and environment and what constituted the particular experiences that led to the 
understandings being assessed. These assessments frequently fail to address the historical 
development and diffusion of knowledge as it occurs, especially when conceived from a 
situativity perspective. Our goal has been to develop a methodology to capture these interactions 
with the goal of tracing the historical emergence and development of knowledge, allowing us to 
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capture knowing-in-the-making. Central to our situative epistemological commitment is the 
conviction that knowing about is a continuous event distributed across multiple time frames and 
environmental particulars — not a static structure existing in an individual's head. As such, it was 
essential to have a methodology for capturing and tracing cognition conceived as such. 

In developing our methodology, we used the structural framework of ANT for the 
development of our analytical approach. We found this approach appropriate because it allows 
researchers to position actors (human or non-human) within the larger context in which they 
function, acknowledging their mediating roles in forming context as well as other actors, 
networks, and outcomes of these interactions. We found such an approach particularly relevant 
to our interest in capturing cognitions conceived of as situated through the relations among task, 
individual, and setting — especially non-human objects (e.g., computers). However, given some 
of the criticisms of ANT more generally, we did not find it particularly useful as a theoretical 
grounding for conceiving the relationships among human and non-human actors. 

In advancing our CNA methodology, we have adhered to the five standards put forth by 
Schoenfeld (1992). We began with a description of the theoretical context, the issues to be 
addressed and the rationale for the method. This was then followed by an in-depth description of 
the method, along with its application to particular data sets. We described its application in 
terms of identifying important interactions and patterns, in comparing frequency of behaviors 
among two or more groups. More importantly, from our perspective, we described and provided 
an in-depth example of how we applied the method to trace the emergence, evolution, and 
diffusion of a particular practice (tilting a planet model on its axis). We then offered 
methodological discussion of the trustworthiness, usefulness, scope of application, and 
limitations of the method. 



Although researchers have typically relied on think alouds, protocol analysis, and 
stimulated recall to understand cognition, the CNA methodology being proposed here has a 
unique opportunity to capture cognitive activity on the fly. The ability to capture knowing-in- 
the-making is becoming increasingly valuable as we see the design and implementation of more 
participatory-based learning environments where students are central initiators in the learning 
process. This information is particularly useful in design experiments where the goal is to 
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introduce innovations and understand the impact of these interventions on environments 
intentionally designed to support learning. In this article, we provided evidence for the 
trustworthiness and usefulness of our CNA methodology. Future research must continue to 
examine the applicability of this approach for multiple contexts. More importantly, we need to 
continue to explore what types of information this method, as opposed to other techniques, 
provides, and why this information is useful. 

We are at a time of paradigmatic shifts in ontology and epistemology and researchers 
need to continue to look for novel techniques that are able to capture cognition conceived as 
situated. As we continue to do research that becomes less exploratory and more confirmatory, 
the descriptions of context, knowing, and the relations among the two will become more refined. 
However, commensurate with recent epistemological shifts, it is a time for real and meaningful 
exploration, applying data analytic techniques that afford researchers rich descriptions of the 
process in which learners become knowledgeably skillful within the context of their 
participation. Once we have a better understanding of these processes, we can then examine the 
influence of various interventions for supporting students in this process. We hope that this 
manuscript stimulates discussion and prompts researchers to explore innovative methods for 
capturing the emergence, evolution, and diffusion of knowing, allowing us to better understand 
cognition and to improve environments intentionally designed to support learning. 
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Footnotes 



' In fact, in our usage, we are simply using ANT as a structural framework for the development of our analytical 
approach and not the theoretical framework for conceiving the relationship among human and nonhuman actors. 

^ These distinctions will become more apparent below, when we apply the CNA methodology to examples from our 
own research. 
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Figure 1 . Web-based coding form used to capture the salient information of a node. 
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Figure 2 . Time duration of Activity Nodes for each initiator. Each time interval on the Y-axis 
represents one minute, and bars represent time duration of each Activity Node. 
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Figure 3 . Activity Nodes for each initiator. For this diagram the bars have been abstracted into 
circles that represent relative position in time. 
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Figure 4. Activity Nodes for each initiator (represented by ovals) with lines corresponding to the 
movement of each individual. For example, we can see that Tom acts as a participant for nodes 
1-3 where Jose is the initiator. At times 4-6, Jose then serves as the primary initiator for each of 
the nodes he is involved. Examination of this figure reveals which individuals took the most 
active role in initiating the interactions and what collaboration occurred in the nodes. 
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Figure 5. Activity Nodes for each initiator with lines representing the movement of each 
individual. In addition dark nodes represent those nodes in which a particular practice is being 
carried out, referred to here as "The Practice." Examination of this network indicates that "The 
Practice" began with Mary who then collaborated with Jose and Tom at time three. At time four 
Mary and Jose then collaborated on "The Practice." At time five Jose then collaborated on "The 
Practice" with Tom and Susie. Susie continued to carry out "The Practice" at time six, while the 
other three students carried out other practices. 
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Figure 6. Network Representing the Diffusion of Task/Goal Definition for Students in the Solar 
and the Theater Groups. Each Box Represents a Node in which Task/Goal Definition was coded 
as a Practice. The row Associated with Each Box Indicates that Student or Mentor was an 
Initiator, with a Node Potentially Having Multiple Initiators. Vertical alignment indicates that 
initiators were collaborating. 
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Figure 7 . Activity Nodes for each initiator with lines representing the movement of each 
individual. 
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Table 1. Summary Labels for the Various Features that Constitute a Node. 



Category 


Description 


Issue at 


a summary label that is chosen to identify the content of the node. It is the “direct” 


Hand 


object of discussion or manipulation (the only way a practice can be considered an 
issue at hand is if it becomes the explicit object of discussion or manipulation). It can 
refer to an artifact, tool-related practice, or a conceptual tool/process. 


Initiator 


an individual, technology, belief, raw material, object, or group (when engaged in a 
practice as a single unit) that is producing an action. Although we listed only 
observable initiators, it is important to note that actors do not emerge in a vacuum; 
rather, they exist within a context that is reciprocally constituted by the cultural 
surround and transformed by their initiator actions. In this fashion the cultural 
surround could arguably be considered an initiator involved in defining the specifics 
of the issue at hand. However, it becomes impossible and we believe overly 
presumptuous to define the numerous aspects of cultural influence that interact with 
the issue at hand. Therefore, we have not included these non-observable (yet 
potentially important factors) in our coding scheme and must acknowledge this as a 
limitation. 


Participant 


an individual who is involved in a node but not initiating the action. 


Resource 


“any piece of information, object, tool, or machine” that an initiator uses to carry out 
a practice (Roth, 1996, p. 191). In addition to technological tools, our definition of 
tool includes those of a conceptual nature (i.e., heat-color relations). An artifact is 
transformed to a resource when it is used by an actor as part of a practice. As such, it 
is important to note that it only becomes a resource within a particular node if it is 
being used by an initiator to support a practice. 


Practice 


an activity that is carried out by an initiator who is using a resource. Practices can be 
tool related (i.e., embodied tool-related laboratory skills), scientific (i.e., calculating), 
instructional related (i.e., coaching), or learning related (i.e., using an inquiry strategy) 
and always involve the use of a resource. 
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Table 2. Classification of Nodes, Categories, Sub-Categories, and Types for Example 1. 



Node 


Issue at hand 


Initiator 


Participant 


Practices 


Resource 


1 


Object: 


Student: 




Tool-Related: 






Earth, Moon, 


Marvin 




View Pt. Setting 






Sun Model 






Modeling: 












Model Building VR 




2 


Concent: 


Student: 


Student: 


Student: 






lunar eclipse 


Marvin 


Keith 


Group Discussion 


St. Props 




Practice: 








(hands) 




model 








Computer 




building VR 








Model 


3 


Object: 


Student: 




Modeling: 






v-pts 


Marvin 




Model Evaluation 


Computer 




Concept: 








model 




lunar eclipse 










4 


Practice: 


Student: 


Student: 


Instr Practice: 






setting v-pts. 


Marvin 


Roger 


questioning 


Computer 








Mentor: 


Student Practice: 


model 








Igor 


question teacher 




5 


Practice: 


Student: 


Student: 


Instr Practice: 






setting v-pts 


Roger 


Marvin 


questioning 


Computer 








Mentor: 


Student Practice: 


model 








Igor 


retelling 




6 


Practice: 


Student: 


Student: 


Modeling Practices: 






setting v-pts. 


Marvin 


Roger 


debugging 


Computer 






Mentor: 




Instr Practice: 


model 






Igor 




coaching 




7 


Practice: 


Mentor: 


Student: 


Instr Practice: 






setting v-pts. 


Igor 


Roger 

Marvin 


Just-in-time lecture 




8 


Obiect: 


Student: 




Tool-related Practice: 






Earth, Moon, 


Marvin 




Setting viewpoints 


Computer 




Sun Model 








model 



C. Rich. 
6 



6 



6 



2 



2 



2 



2 

4 
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Appendix A 



Scenario II (Line of Nodes) : In this scenario, one student (Erin) has developed a Earth, Moon, Sun model 
that includes a visualization of the line of nodes. We will pick up the interaction as a student from 
another group (Marvin) comes over to Erin’s group for help. 

Participant Description 

Marvin 



Erin 

Erin 

Marvin 

Erin 

Marvin 

Erin 

Marvin 

Erin 



There is a rumor that you’re working on the line of nodes in your model. I’m not 
sure what it is 

It’ s where the plane of the ecliptic between the sun and earth, and the plane of 
the earth and the moon intersects. . . [Marvin nods] 

[pointing to the screen, Erin continues] The way I made mine, I made a long 
cylinder and made it a very long line. 

Wow, that thing is a cylinder! 



Yeah, (pointing to a line on the screen).... I grouped the earth and the line of 
nodes so the line of nodes would stay with the earth when it revolves. 

That’s a good idea. So what you are trying to demonstrate here is when the line 
of nodes come together . . . That’ s when the eclipse happens .... That’ s 
good... Wow! 

Yeah this is going to be neat when it works. When I did it last time I grouped it 
wrong so be careful. 

Thanks! 

[She colors the line of nodes in her model chose a shade of green that would be 
visually eye catching] 



Node 

1 



Later the scenario continues when Erin presents her model to explain the difference between a solar 
eclipse and a lunar eclipse. 

Participant Description Node 

Erin 



You can only have total eclipses when the moon is on the line of nodes. If the 
moon is on the side of the earth facing the sun, that would be a new moon, you 
can get a solar eclipse because the sun would be blocked by the moon’s shadow. 
And when the moon is on the side of the earth, that would be a full moon, you 
can get lunar ellipses because the moon passes through the earth’s shadow 
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Appendix B 

Scenario III (The Black Boxing of Logarithms') : Keith realizes that scaling is going to become an 
important issue in his solar system model. He shares his thought with Marvin. 



Particinant 


Descriotion 


Node 


Keith 


We have to think about scale 


1 


Marvin 


I have no idea to what the actual scale is. . ..(is it) like what sizes are relative to 
each other? 




Keith 


If this is Earth [finds a dead ladybug on the desk and brings it into Marvin *s 
attention] the sun is. . .the sun is uh. . . 200ft away [has his arms wide open] 




Marvin 


200ft away? 




Keith 


That’s the scale, we can’t demonstrate it on the monitor, so we have to rotate on 
an orbit, it’s too small [pointing to the bug]. 




Marvin 


I will let you handle that. 




Keith 


[Based on a question by the researcher, Keith states that he has decided to use 


2 


(narrative) 


natural logarithms to scale his model of the solar system so you can see all 
objects at the correct sizes and distances on one screen] 




Keith 


[Keith takes out his calculator, looks up planet sizes and distances for his book, 


3 


(narrative) 


punches numbers into his calculator, and creates a logarithm table of planet 
sizes and distances] 




Keith 

(narrative) 


[Keith then uses the logarithm table to create objects on the computer] 


4 


Marvin 


[Marvin takes logarithm table the proceeds to use the numbers to size and 


5 


(narrative) 


position the planets] 




Keith 


We have to think about scale. 


6 


Keith 


Why are you putting in those numbers? [points to the computer screen] 




Marvin 


Because I am setting up the scale of my solar system. 




Keith 


Yeah, but why do those numbers work so well? 




Marvin 


I don't know. You used some logarithm thing that will make size and distance 
all on the same scale. 




Keith 


Logarithms are special numbers that . . . [Marvin cuts him off] 




Marvin 


I don't understand it . . . but you did it (he turns back to the computer). 




Marvin 


[Marvin continues to use the numbers taken from the table, and makes spheres of 


7 


(narrative) 


the correct distance and sizes] 
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Appendix C 



Scenario IVLtEvaluating the Project) : The next scenario picks up towards the end of the first project of the 
course, the creation of a Celestial Sphere. We begin with Ralph and Mary: 



Participant 
Ralph & 
Mary 
(narrative) 

Mary 



Ralph 

(narrative) 

Ralph 

Mary 

Ralph 

Ralph 

Igor 



Mary 

Igor 



Description 



Node 



[They are looking at the Celestial Sphere that they have constructed on the 
screen (the Equator, the Tropic of Capricorn, the Tropic of Cancer, and the 
earth’s axis to line up). They look at it from multiple angles to check it for 
accuracy.] 

Maybe we should go back and check to see if we have all the features that are 
supposed to be in the model, [picking up the assignment sheet]. Let’s see, the 
equator. . .got that. . .the Tropics. . .got those. . . 

[Ralph, listening to Mary, moves to his computer and opens Netscape. He goes 
to the course web site and looks for the project link, and begins looking at the 
resources available for this project.] 

Ho Ho! Here’s what he has on the web. A bunch of labels. An earth with the 
tropic of Cancer and the Tropic of Capricorn on it. 

Oh yeah? That’s probably what I want. You mean that it’s one of those, it shows 
you how to do it?... 

Yeah, it’s on the web. Textures. . . labels, all stuff for our model. 

[to Igor] What would we do with these? 

Those could be textures for the sun. And so, the sun at certain points, creates 
these wonderful little points that we want you guys to be able to demonstrate 
with your model. And so maybe you put the sun here, [pointing to where the 
label would go on their model] and you can put a texture on it, so that way, if 
somebody comes to the web site, which they will do, because this is such a cool 
class, and that way you can know what’s happening, because the sun will have 
vernal equinox on it. 

Because what will? I’m sorry 

If you put the sun here on the vernal equinox and you wrap the texture around it, 
just like you’ve done with the Earth, that way the sun will say, oh that’s the 
vernal equinox, that way someone can easily describe what’s going on with your 



1 



2 



3 



4 



5 
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Mary 

Igor 

Mary 

Igor 



Mary 



Igor 

Mary 



model to someone who doesn’t have a clue. 

Cool. So that you’d be getting that just by using that texture, then that has just 
about everything we need 

Or, you could also do a labeled text. You could take a little arrow and draw a 
point that go that’s the vernal equinox. There’s tons of ways to demonstrate it. 
Sweet. . .Will we be looking at these projects in the CAVE? 

Yes, we may not view the Celestial Sphere because some of the stuff we’re 
doing here isn’t really implemented in the CAVE yet. You can’t read text in the 
CAVE, and, uh, transparencies don’t work in the CAVE, but in the next project, 
the Earth moon, sun system, everything we’ll be doing there will be viewable in 
the CAVE. You can go you can stand at the moon, and look at the sun. And so 
on... 

You know what I was thinking of doing... it will probably be a waste then 
because it will be much more simpler there. I’m trying to do a make a cylinder to 
do the 23.5 degree slant. I was going to use the angle. . . Will that be too 
complicated. 

Oh no... that’s a good idea. But there’s a slightly easier way to do it. 

I need to line this up though, let’s see. How did I do that. ..[Mary turns to the 
computer, and calls up the part of the project she was working on.] 



6 



7 



8 



9 
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